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ABSTRACT 


In  the  ongoing  attempt  to  deliver  quality  goods  and  services  in  less  time  and  cost 
to  the  Government,  the  concept  of  “Best  Value  “  contracting  has  been  explored.  One 
element  of  best  value  contracting  is  to  consider  factors  along  with  price  in  the  source 
selection  process  to  achieve  a  perceived  best  value  to  the  Government.  Past  performance 
measurement  and  its  use  in  best  value  procurements  has  been  developed  since  the  early 
1990s. 

This  thesis  explores  the  past  performance  guidance  within  DoD  in  its  use  as  a 
source  selection  factor.  Automated  tools  for  tracking  past  performance  will  be  examined 
as  well  as  their  effectiveness  and  problems  associated  with  data  collection  and  use. 
Several  steps  must  occur  prior  to  past  performance  playing  the  role  that  it  was  envisioned 
with  its  inclusion  as  a  factor  along  with  price  in  the  source  selection  process. 

So  that  past  performance  data  can  be  used  more  effectively,  this  thesis 
recommends  that  contracting  personnel  and  contractors  establish  a  process  that 
standardizes  the  method  of  measurement,  automates  its  collection,  and  allows  for  viewing 
across  the  spectrum  of  past  performance  data.  Furthermore,  past  performance  data 
should  be  used  within  a  risk  management  framework  that  assesses  a  contractor’s 
capability  risk  to  perform  on  the  current  procurement. 
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I.  INTRODUCTION 


A.  PURPOSE 

This  thesis  researches  and  investigates  the  use  of  past  performance  within  the 
Department  of  the  Army.  Specifically,  current  guidance  on  the  use  of  past  performance 
will  be  looked  at  and  methods  taken  to  implement  and  comply  with  the  guidance  will  be 
identified.  The  reasons  why  past  performance  is  considered  and  who  uses  this  data  will 
also  be  discussed.  Problems  encountered  in  the  implementation  of  the  current  guidance 
will  be  discussed  as  well  as  the  effectiveness  of  the  current  guidance  and  approach. 
Finally,  recommendations  for  more  effective  use  of  past  performance  in  the  source 
selection  process  will  be  made. 

Considerable  time  and  money  has  gone  into  the  collection  and  use  of  past 
performance  within  DoD  and  the  Army.  Its  use  has  had  mixed  results,  and  many  within 
DoD  are  questioning  the  effectiveness  of  past  performance  and  its  role. 

B,  BENEFITS  OF  THE  RESEARCH 

This  thesis  will  assess  the  effectiveness  of  the  use  of  past  performance  within  the 
Department  of  the  Army.  With  a  better  understanding  of  the  current  problems  associated 
with  the  use  of  past  performance,  more  effective  strategies  can  be  developed  for  its  use  in 
the  future. 

C.  RESEARCH  OBJECTIVE 

The  primary  objective  of  this  thesis  is  to  provide  an  assessment  of  current 
problems  associated  with  the  use  of  past  performance  and  identifies  more  effective  ways 
for  its  use. 

D,  RESEARCH  QUESTIONS 

1,  Primary  Research  Question 

How  can  the  Army  best  use  past  performance  in  the  source  selection  process? 

2,  Secondary  Research  Questions 

•  What  does  past  performance  actually  represent? 

•  How  is  past  performance  used  today? 

•  What  guidance  currently  exists  on  past  performance? 

•  What  organizations  gather  and  use  past  performance  data? 
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•  What  automated  databases  and  tools  are  used? 

•  What  is  the  overall  eomplianee  with  past  performanee  guidanee? 

•  What  are  some  possible  methods  of  past  performanee  use  in  the  future? 

E.  SCOPE 

Current  guidanee  on  the  use  of  past  performanee  will  be  looked  at  and  methods 
taken  to  implement  and  eomply  with  the  guidanee  will  be  identified.  Automated  tools  for 
traeking  past  performanee  will  be  explored  as  well  as  their  effeetiveness  and  problems 
eneountered  by  users.  A  diseussion  will  be  presented  as  to  the  best  fit  for  past 
performanee  in  the  award  proeess  and  enhaneements  required  to  fully  implement  the 
eurrent  guidanee.  The  value  of  past  performanee  considerations  in  the  source  selection 
process  will  also  be  discussed. 

F.  METHODOLOGY 

This  research  uses  the  following  data  sources:  a  comprehensive  review  of  the 
guidance  within  Department  of  Defense,  interviews  with  individuals  engaged  in  the 
process,  journal  articles,  technical  reports,  and  organizational  experiences  in  past 
performance  use  and  implementation. 

•  A  comprehensive  review  of  all  major  guidance  on  past  performance  will 
be  performed. 

•  Telephone  interviews  will  be  utilized  to  assess  the  current  compliance 
with  the  guidance  on  past  performance.  Problems  and  current  uses  of  past 
performance  will  be  identified. 

•  The  information  from  the  review  of  current  guidance  on  past  performance 
and  its  current  implementation  obtained  from  interviews  and 
organizational  experience  will  be  analyzed.  Problems  with  current  past 
performance  use  will  be  identified.  Consolidation  of  individual 
recommendations  from  interviews  will  be  formulated. 

G.  ORGANIZATION  OF  THE  THESIS 

The  thesis  is  organized  as  follows: 

•  Chapter  1.  Introduction 

•  Chapter  II.  Provides  a  background  of  past  performance  use  and  a  review 
of  past  performance  guidance  and  literature. 
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•  Chapter  III.  Explains  the  methods  of  data  colleetion  used  in  gathering  past 
performanee  information,  and  identification  problems  and  challenges 
associated  with  the  current  use  of  past  performance  information.  This 
chapter  also  provides  an  analysis  of  the  data. 

•  Chapter  IV.  Draws  conclusions  based  on  the  analysis  and  provides 
recommendation  for  future  use  of  past  performance. 
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II.  CURRENT  GUIDANCE  ON  PAST  PERFORMANCE 


A.  INTRODUCTION 

This  chapter  discusses  the  current  DoD  guidanee  with  regards  to  past  performanee 
as  well  as  the  implementation  of  data  gathering  and  the  use  of  the  data  within  the  source 
seleetion  proeess. 

Past  Performance  Information  (PPI)  guidance  was  first  outlined  in  OFPP  Poliey 
Letter  92-5  in  Deeember  1992.  [Ref  17]  Attempts  to  introduee  past  performanee 
information  into  source  seleetion  early  in  aequisition  reform  were  deseribed  in  the  ‘Final 
Report  for  the  Contraetor  Past  Performanee  System  Evaluation  Study  to  the  Deputy 
Under  Seeretary  of  Defense  (Aequisition  Reform)  in  June  of  1996.’  This  report  depieted 
eontinuing  obstaeles  to  the  use  of  past  performanee  information  in  souree  seleetion 
proeess.  Major  findings  of  the  eited  report  are  as  follows  below: 

•  Weak  and  inaeeurate  quality  and  delivery  data  proeesses 

•  Laek  of  tools  to  eolleet  aeeurate  data 

•  Time  to  validate  performanee  data 

•  Impact  on  the  aequisition  streamlining  efforts  to  reduced  proeurement 
administrative  lead  times  (PALT) 

•  Produetivity  impaets 

•  Administrative  burden 

•  Laek  of  experienee  with  subjective  deeision  making 

•  Laek  of  tools  to  eolleet  aeeurate  data 

•  Low  bidder  mind  set 

•  Risk  avoidance  culture  [Ref  5:p.  38] 

The  use  of  past  performanee  information  has  always  been  a  part  of  the 
Government’s  proeurement  proeess.  Historieally,  contraetor’s  past  performanee  was 
eonsidered  in  making  a  responsibility  determination  as  to  the  eontractor’s  ability  to 
perform  on  a  eurrent  eontraet.  Lor  eaeh  proeurement  aetion,  the  eontraeting  offieer  must 
make  a  determination  if  the  eontraetor  is  responsive  to  the  request  for  proposal  and  if  he 
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is  responsible.  This  determination  of  responsibility  primarily  is  based  on  the  contractor’s 
delivery  and  quality  performance.  A  preaward  survey  is  the  most  common  tool  to  assist 
the  contracting  officer  in  his  determination. 

In  the  early  1990s  past  performance  was  elevated  to  be  a  separate  factor  along 
with  price,  and  other  factors,  for  the  purposes  of  determining  the  “best  value”  for  the 
Government.  The  focus,  therefore,  shifted  to  one  of  not  just  responsibility  but  one  of 
levels  of  responsibility  of  any  given  contractor.  In  theory,  only  responsible  contractors 
should  be  evaluated  for  best  value  purposes.  The  main  idea  behind  making  past 
performance  a  factor  along  with  price  is  that  a  contractor’s  past  performance  reflects 
varying  degrees  of  risks  even  though  he  would  be  determined  responsible  for  any  given 
contract  action. 

Past  performance  has  two  facets  within  DoD:  one  of  data  gathering  and  the  other 
on  the  implementation  or  use  of  the  data  gathered.  The  collection  of  past  performance 
data  can  be  accomplished  using  several  methods,  but  it  is  ultimately  the  responsibility  of 
the  contracting  office  to  gather  the  necessary  data  to  make  supportable  conclusions.  The 
primary  sources  of  past  performance  data  come  from  existing  databases  that  gather  past 
performance  information  on  deliveries  and  quality.  Past  performance  information  can  be 
and  is  often  obtained  as  part  of  the  proposal  process.  In  order  for  past  performance  to  be 
used  effectively,  credible  and  complete  data  is  required  on  a  contractor’s  performance. 
The  second  facet  of  the  use  of  past  performance  within  the  source  selection  process  is 
how  and  when  the  information  is  used.  Issues  such  as  relevance  of  the  data  to  the  current 
procurement  as  well  as  the  time  frame  of  the  performance  history  must  be  taken  into 
consideration. 

B,  DOD  GUIDANCE 

The  Office  of  Federal  Procurement  Policy  (OFPP)  Letter  92-5  (issued  December 
30  1992)  required  that  past  performance  be  evaluated  in  all  competitive  negotiated 
contracts  over  $100,000.  The  letter  stated: 

Past  performance  is  relevant  information  regarding  a  contractor’s  actions 
under  previously  awarded  contracts.  It  includes  the  contractor’s  record 
and  conforming  to  specifications  as  to  standards  of  good  workmanship; 
the  contractor’s  record  of  containing  and  forecasting  cost  on  any  cost 
reimbursable  contracts;  the  contractor’s  adherence  to  the  contract 
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schedules,  ineluding  the  administrative  aspeets  of  performanee;  the 
eontraetor’s  history  for  reasonable  and  eooperative  and  eommitment  to 
eustomer  satisfaetion;  generally,  the  eontractor’s  business-like  eoneern  for 
the  interest  of  the  customer.  [Ref  17] 

This  poliey  letter  does  allow  for  waiver  if  it  is  determined  by  the  eontraeting 
offieer  that  past  performanee  inclusion  as  a  selection  factor  is  not  appropriate.  Sueh 
determination  must  be  in  writing  and  ineluded  in  the  eontract  file. 

In  most  oases,  this  determination  is  based  on  the  eontraeting  offieer’ s  deoiding 
there  is  no  signifioant  differenoe  in  the  past  performanee  of  the  oontractors  who  will  bid 
on  the  solioitation.  This  ooours  most  often  when  the  proourements  are  restricted  to  other 
than  full  and  open  competition  and  the  eontraeting  officer  knows  the  oontraotors  who  are 
eligible  to  bid. 

Federal  Aoquisition  Regulations  (FAR)  15.304  delineates  the  requirements  of 
when  and  how  past  performanee  information  is  to  be  used.  FAR  42.15  requires  that 
information  be  eolleeted  for  use  in  the  evaluation  proeess.  FAR  12.206  addresses  use  of 
past  performance  information  for  eommereial  items. 

12.206  —  Use  of  Past  Performance. 


Past  performanee  should  be  an  important  element  of  every  evaluation  and 
eontract  award  for  commercial  items.  Contraeting  offieers  should  eonsider 
past  performanee  data  from  a  wide  variety  of  sources  both  inside  and 
outside  the  Federal  Government  in  aeeordanee  with  the  policies  and 
proeedures  eontained  in  Subpart  9.1,  13.106,  or  Subpart  15.3,  as 
applieable. 

The  use  of  past  performanee  outlined  in  OFPP  Poliey  Letter  92-5  (issued 
Deeember  30  1992)  was  amended  in  the  FAR  15  rewrite,  published  January  1,  1998.  This 
letter  states  that: 

•  Past  performanee  shall  be  evaluated  in  all  source  seleetions  for  negotiated 
eompetitive  aequisitions  issued  on  or  after  January  1,  1999,  for 
aequisitions  expeeted  to  exeeed  $100,000. 

•  Past  performance  need  not  be  evaluated  if  the  eontraeting  offieer 
doeuments  the  reason  past  performanee  is  not  an  appropriate  evaluation 
faetor  for  the  aequisition.  [Ref.  17] 
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In  the  January  1999  Deviation,  the  Offiee  of  the  Under  Seeretary  of  Defense 
(Acquisition  and  Technology)  allowed  DoD  contracting  activities  to  change  several 
levels  at  which  PPI  was  used  and  collected. 

It  raised  the  requirement  for  use  on: 

•  System  and  operational  support  procurement  to  $5,000,000. 

•  Services,  information  technology,  or  science  and  technology  expected  to 
exceed  $1,000,000. 

•  Fuels  and  health  care  expected  to  exceed  $100,000. 

This  deviation  also  changed  the  threshold  at  which  contractor  past  information 
was  to  be  collected: 

•  Systems  and  operations  support  contracts  to  $5,000,000 

•  Services  and  information  technology  contracts  to  $1,000,000 

•  Fuel  and  heath  care  contracts  to  $100,000 

•  No  threshold  for  science  and  technology  contracts  [Ref  2:p.  2] 

FAR  42.15  delineated  what  type  of  Past  Performance  Information  (PPI)  should  be 
collected,  including  the  ground  rules  for  its  maintenance  and  the  contractors’  input  into 
the  process.  It  states  that  all  relevant  information  on  a  contractor’s  past  performance 
should  be  gathered  to  include  the  contractor’s  conformance  to  contract  requirements, 
standards  of  good  workmanship,  ability  to  control  cost,  adherence  to  contract  schedules, 
history  of  cooperation  and  commitment  to  customer  satisfaction.  The  primary  thrust  of 
FAR  42.15  is  to  allow  for  contractor  review  of  any  past  performance  information  (PPI) 
collected  by  an  agency.  The  guidance  outlines  that  a  contractor  is  to  be  given  a  minimum 
of  30  days  to  review  any  data  collected  and  comment,  rebut  or  provide  additional 
information  about  their  performance  record  (FAR  42.1503b).  Failure  to  respond  within 
the  30-day  period  will  be  taken  as  agreement  with  the  assessment.  The  Contracting 
Officer  may  grant  an  extension  to  the  30-day  review  period.  [Ref  11] 

Any  disagreement  on  the  data  gathered  between  the  evaluators  and  the  contractor 
will  be  resolved  at  one  level  above  the  evaluator,  and  ultimate  determination  on  the 
performance  record  lies  with  the  contracting  activity.  The  guidance  also  states  that  the 
data  should  not  be  retained  for  more  than  three  years  after  completion  of  a  contract.  [Ref 
10] 
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C.  IMPLEMENTATION 

The  collection  and  use  of  past  performance  data  comes  from  several  different 
sources.  Databases  such  as  Past  Performance  Information  Management  System  (PPIMS) 
provides  information  across  the  Army  on  high  dollar  contracts.  Each  command  also 
maintains  an  internal  tracking  system  for  such  things  as  on-time  delivery  and  quality 
performance.  These  vary  in  sophistication  and  completeness  among  commands.  Another 
primary  source  is  past  performance  information  requested  during  the  solicitation  process 
from  each  offeror.  This  information  then  is  used  along  with  additional  information 
gathering  such  as  interviews  and  surveys  with  past  customers. 

Collection  of  past  performance  data  faces  several  inherent  difficulties  based  on 
the  size  of  the  Federal  procurement  system  and  the  variety  and  number  of  actions.  The 
major  issues  related  to  the  collection  of  Past  Performance  Information  (PPI)  are  as 
follows: 

•  Capturing  all  relevant  PPI  in  a  manner  that  is  useable  for  evaluation 
purposes. 

•  Uniformity  of  information  gathered. 

•  Automation  tools  available  for  collection. 

•  Determining  who  gathers  the  information. 

•  Measurement  or  metric  used  (grading  system). 

•  Deciding  how  long  information  is  retained,  in  what  form,  and  by  whom. 

•  Assessing  whether  the  cost  of  collection  outweigh  the  benefits. 

Data  collection  methods  and  processes  were  left  up  to  the  individual  agencies  and 
services.  The  Army  initially  relied  on  the  Contractor  Information  System  (CIS)  database, 
which  holds  information  on  a  contractor’s  performance.  This  database  lacked 
completeness  and  consistency  of  data.  Data  was  often  incomplete  or  with  no  information 
about  contract  performance.  The  information  in  the  system  was  often  inputted  by  the 
same  individuals  who  were  using  the  data  and  represented  no  new  information  for  their 
analysis. 

The  Army  currently  uses  PPIMS,  a  web  based  information  system.  Under 
PPIMS,  information  is  gathered  on  Performance  Assessment  Reports  (PARs),  which  are 
initiated  and/or  modified,  via  a  data  input  screen.  PARs  are  prepared  upon  completion  of 
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the  contract.  When  the  contract  period  of  performance  does  not  exceed  18  months,  a 
single  final  assessment  report  is  submitted.  When  contract  performance  is  expected  to 
exceed  18  months,  an  interim  report  is  completed  at  12  months  and  annually  thereafter 
until  contract  is  physically  complete.  An  out-of-cycle  addendum  report  can  be  prepared 
if  there  is  a  need  to  document  an  extraordinary  event  prior  to  the  required  reporting 
period.  At  submission  of  an  evaluation  the  contractor  has  the  ability  to  provide  a  rebuttal 
to  the  information.  Once  that  process  has  been  accomplished,  the  data  is  retained  to  be 
available  for  use  in  making  future  award  decisions.  PPl  is  retained  in  the  PPIMS  for  three 
years  after  contract  completion.  The  official  signed  hard  copy  is  retained  in  the  official 
contract  file.  [Ref  21] 

The  PAR  report  is  required  at  the  following  thresholds,  as  required  by  AFARS 
5142.1502-90; 


Business  Sector 

Threshold 

Systems 

$5M 

Operation  Support 

$5M 

Services 

$1M 

Information  Technology 

$1M 

Construction 

$500,000 

Architect-Engineering 

$25,000 

Assessing  Official 

PM 

Contracting  Officer 
Contracting  Officer  or  PM 
Contracting  Officer 
lAW  FAR/DFARS/AFARS  Parts 
36/236/5136 

lAW  FAR/DFARS/AFARS  Parts 
36/236/5136  [Ref.  liParagraph  a] 


PPIMS  looks  at  and  rates  several  areas  within  a  contractor’s  past  performance. 

Data  is  captured  in  the  following  areas: 

•  Quality  of  Product  or  Service  -  Contractor’s  conformance  to  contract 
requirements,  specifications,  and  standards  of  good  workmanship  (e.g., 
commonly  accepted  technical,  professional,  environmental,  or  safety  and 
health  standards.) 

•  Schedule  -  Timeliness  of  the  contractor  against  the  completion  of  the 
contract,  task  orders,  milestones,  delivery  schedules,  administrative 
requirements  (e.g.,  efforts  that  contribute  to  or  effect  the  schedule 
variance.) 

•  Cost  Control  -  (Not  required  for  Firm-Fixed  Price  and  Firm-Fixed  Price 
with  Economic  Price  Adjustment  contracts.)  Contractor’s  effectiveness  in 
forecasting,  managing,  and  controlling  contract  cost. 
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•  Business  Relations  -  Integration  and  coordination  of  all  activity  needed  to 
execute  the  contract,  specifically  the  timeliness,  completeness  and  quality 
of  problem  identification,  corrective  action  plans,  proposal  submittals,  the 
contractor’s  history  of  reasonable  and  cooperative  behavior,  customer 
satisfaction,  timely  award  and  management  of  subcontracts,  and  whether 
the  contractor  met  small/small  disadvantaged  and  women-owned  business 
participation  goals. 

•  Management  of  Key  Personnel  -  (For  services  and  information 
technology  contracts  only)  -  Contractor’s  performance  in  selecting, 
retaining,  supporting,  and  replacing,  when  necessary,  key  personnel.  [Ref 
21] 

The  PPIMS  rating  system  described  below  is  used  to  assess  contractor 
performance  for  all  applicable  PPI  elements: 

•  Exceptional  (Dark  Blue)  -  Performance  meets  contractual  requirements 

and  exceeds  many  to  the  Government’s  benefit.  The  contractual 

performance  of  the  element  or  sub-element  being  assessed  was 
accomplished  with  few  minor  problems  for  which  corrective  actions  taken 
by  the  contractor  were  highly  effective. 

•  Very  Good  (Purple)  -  Performance  meets  contractual  requirements  and 
exceeds  many  to  the  Government’s  benefit.  The  contractual  performance 
of  the  element  or  sub-element  being  assessed  was  accomplished  with  some 
minor  problems  for  which  corrective  actions  taken  by  the  contractor  were 
effective. 

•  Satisfactory  (Green)  -  Performance  meets  contractual  requirements.  The 
contractual  performance  of  the  element  or  sub-element  contains  some 
minor  problems  for  which  corrective  actions  taken  by  the  contractor 
appear  or  were  satisfactory. 

•  Marginal  (Yellow)  -  Performance  does  not  meet  some  contractual 
requirements.  The  contractual  performance  of  the  element  or  sub-element 
being  assessed  reflects  a  serious  problem  for  which  the  contractor  has  not 
yet  identified  corrective  actions.  The  contractor’s  proposed  actions  appear 
only  marginally  effective  or  were  not  fully  implemented. 

•  Unsatisfactory  (Red)  -  Performance  does  not  meet  most  contractual 
requirements  and  recovery  is  not  likely  in  a  timely  manner.  The 
contractual  performance  of  the  element  or  sub-element  contains  serious 
problem(s)  for  which  the  contractor’s  corrective  actions  appear  or  were 
ineffective.  [Ref  1  iParagraph  g] 

The  methodology  for  rating  past  performance  can  include  color  coding,  adjectival, 
or  numerical  rating  systems,  and  plus  or  minus  checks.  What  is  important  is  not  the 
rating  methodology,  but  the  consistency  with  which  it  is  applied  to  elements  of  proposals. 
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Consistency  must  be  maintained  not  only  in  the  application  of  source  selection  plan  but 
also  among  contractors  to  ensure  a  thorough  and  fair  evaluation.  Each  rating  should  be 
supported  by  sufficient  rationale  to  allow  a  third  party  to  draw  a  similar  conclusion. 

The  number  of  rating  levels  can  also  impact  the  effectiveness  of  the  evaluations. 
Five  rating  levels  allows  for  a  wide  enough  rating  spread  to  make  the  necessary 
distinctions  in  the  scoring  or  past  performance  data  and  in  the  final  rating  of  contractors 
past  performance  during  the  source  selection  process.  Five  levels  of  ratings  is  supported 
and  recommended  by  USD(A&T)  memorandum  for  the  past  performance  OIPT,  dated 
August  11,  1997.  [Ref  14] 

Another  aspect  of  the  rating  process  is  what  value  is  considered  average.  The 
DoD  Guide  to  Collection  and  Use  of  Past  Performance  Information  suggests  the  “^4 
fundamental  principle  for  rating  is  that  contractors  shall  not  be  assessed  below  a  rating 
of  “satisfactory”  for  not  performing  beyond  the  requirements  of  the  contract.  ”  [Ref  9;p. 

5] 

D,  PAST  PERFORMANCE  DATABASES 

Past  performance  as  an  evaluation  factor  is  a  requirement  in  competitive 
selection.  The  methods  of  measuring  past  performance  are  varied  and  in  many  cases 
cumbersome  to  use.  One  of  the  fundamental  problems  with  the  evaluation  of  past 
performance  has  been  that  there  is  not  one  reliable  database  from  which  to  obtain  past 
performance  data;  each  service  has  its  own  data  stored  in  a  variety  of  ways  and  measured 
by  different  methods.  For  major  procurements  that  involve  source  selection  teams,  these 
shortcomings  can  be  mitigated  by  manpower  that  can  gather,  evaluate,  and  validate  the 
accuracy  of  the  data  for  use  in  the  selection  decision.  On  smaller  procurements  under 
$1M,  the  resources  to  apply  to  these  evaluations  are  limited  and  the  time  frames  to  make 
the  awards  are  shorter. 

The  recently  deployed  Past  Performance  Information  Retrieval  System  (PPIRS) 
and  PPIMS  are  efforts  to  overcome  some  of  the  data  collection  problems  encountered  and 
are  a  positive  step  in  a  centralized,  uniform  collection  process.  PPIRS,  an  upgraded 
version  of  the  Past  Performance  Automated  Information  System  (PPAIS),  provides  a 
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central  data  repository  for  past  performanee  data  within  DoD.  The  PPIRS  eentral 
warehouse  allows  retrieval  of  past  performanee  assessment  reports  reeeived  from  four  of 
the  reeognized  Federal  report  eard  colleetion  systems,  which  are: 

•  National  Institutes  of  Health  (NIH)  Contraetor  Performanee  System 
(CPS); 

•  National  Aeronautics  and  Space  Administration  (NASA)  Past 
Performanee  Data  Base  (PPDB); 

•  Army's  Past  Performanee  Information  Management  System  (PPIMS) 

•  Contraetor  Performanee  Assessment  Reporting  System  (CPARS)  used  by 
the  Navy,  USMC,  Air  Foree,  DLA,  and  other  defense  ageneies.  [Ref  20] 

E,  USE  OF  PAST  PERFORMANCE  DATA 

The  use  of  past  performanee  during  the  source  selection  process  has  its  challenges 
even  if  the  information  eollection  has  been  performed  effeetively.  Many  issues  must  be 
addressed  to  achieve  an  effeetive  use  of  past  performance.  How  do  the  evaluators 
evaluate  eontraetors  with  no  past  performanee?  Issues  such  as  recency  of  the  past 
performanee  data  and  relevaney  must  be  addressed  during  the  evaluation  proeess. 
Consisteney  between  evaluators  as  to  how  the  information  is  graded  must  be  monitored. 
The  weighting  that  past  performance  earries  in  relationship  to  other  faetors  is  also 
important  in  that  if  enough  weight  is  not  provided,  the  effeet  on  the  final  seleetion  is 
minimized.  Time  allotted  for  the  evaluation  proeess  is  often  a  eoneern  as  there  is 
eonsiderable  pressure  to  reduce  over  all  proeurement  lead-times.  The  proper  use  and 
evaluation  of  data  from  outside  the  Government’s  past  performanee  reeords  must  be 
elosely  monitored  for  biases  and  eonsistency  with  internal  information.  It  is  important 
that  skilled  and  trained  personnel  perform  the  evaluation  for  eonsistency  of  application. 
Legal  oversight  into  the  proeess  ean  also  be  problematie  if  the  lawyers  unduly  infiuenee 
the  evaluators  into  neutral  or  risk  adverse  seoring. 

The  FAR  provides  some  insight  into  PPI  use  in  the  following  parts: 

15.305  —  Proposal  Evaluation 

(2)  Past  performanee  evaluation 

(i)  Past  performanee  information  is  one  indieator  of  an 
offeror’s  ability  to  perform  the  eontraet  sueeessfully.  The 
currency  and  relevance  of  the  information,  souree  of  the 
information,  eontext  of  the  data,  and  general  trends  in 
eontractor’s  performance  shall  be  eonsidered.  This 
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comparative  assessment  of  past  performanee  information  is 
separate  from  the  responsibility  determination  required 
under  subpart  9.1 

(ii)  The  solicitation  shall  describe  the  approach  for  evaluating 
past  performanee,  ineluding  evaluating  offerors  with  no 
relevant  performanee  history,  and  shall  provide  offerors  an 
opportunity  to  identify  past  or  eurrent  eontraets  (ineluding 
Federal,  State,  and  loeal  Government  and  private)  for 
efforts  similar  to  the  Government  requirement.  The 
solieitation  shall  also  authorize  offerors  to  provide 
information  on  problems  encountered  on  the  identified 
eontraets  and  the  offeror  eorreetive  notions.  The 
Government  shall  oonsider  this  information,  as  well  as 
information  obtained  from  any  other  souroes,  when 
evaluating  the  offeror  past  performanee.  The  souroe 
seleotion  authority  shall  determine  the  relevanoe  of  similar 
past  performanee  information. 

(iii)  The  evaluation  should  take  into  aooount  past  performanee 
information  regarding  predeoessor  oompanies,  key 
personnel  who  have  relevant  experienoe,  or  subeontraetors 
that  will  perform  major  or  eritieal  aspeets  of  the 
requirement  when  sueh  information  is  relevant  to  the 
instant  aequisition. 

(iv)  In  the  ease  of  an  offeror  without  a  reeord  of  relevant  past 
performanee  or  for  whom  information  on  past  performanee 
is  not  available,  the  offeror  may  not  be  evaluated  favorably 
or  unfavorably  on  past  performanee. 

(v)  The  evaluation  should  inelude  the  past  performanee  of 
offerors  in  eomplying  with  subeontraeting  plan  goals  for 
small  disadvantaged  business  (SDB)  eoneems  (see  Subpart 
19.7),  monetary  targets  for  SDB  partieipation  (see 
19.1202),  and  notifications  submitted  under  19.1202-4(b). 

The  relative  ranking  of  past  performanee  data  and  other  factors  to  be  used  in  the 
souree  seleetion  proeess  need  to  be  identified  to  the  contractor  in  the  solieitation  stage  of 
the  proeess.  Seetion  L,  of  the  solicitation  outlines  the  instruetions,  eonditions  and  other 
notiees  to  offers,  the  relative  importanee  of  past  performanee  should  be  deseribed  as 
assoeiated  with  other  faetors  and  priee.  The  solieitation  provides  a  deseription  of  the  data 
to  be  evaluated  and  how  it  is  weighted  relative  to  other  faetors  sueh  as  priee  in  Seetion  M, 
Evaluation  Factors  for  Award.  [Ref.  10:p.  10] 
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Also,  contractors  should  be  afforded  an  opportunity  to  submit  past  performanee 
data  as  part  of  their  proposal.  This  information  ean  be  used  to  validate  the  information  in 
the  PPIMs  databases  as  well  as  gain  new  insights  into  the  contraetor’s  performance. 
Often  on  large  proeurements  self-divulgence  of  a  eontractor’s  own  past  performanee 
history  ean  provide  insights  into  the  eontraetor’s  willingness  and  forthrightness  in  the 
proposal  proeess. 

F.  SUMMARY 

In  summary,  past  performance  data  has  many  faeets  that  must  be  properly 
addressed  for  its  effeetive  use.  A  myriad  of  databases  ean  be  used  and  the  information 
must  be  understood  as  to  how  and  what  is  measured.  Contractor  verification  of  this 
information  must  be  made  prior  to  its  use  in  the  souree  seleetion.  The  weighting  that  past 
performanee  earries  in  the  souree  seleetion  deeision  must  be  eonsidered  and  appropriately 
used  to  have  the  desired  impaet.  All  these  faetors  surrounding  the  use  of  past 
performanee  data  must  be  understood  and  orehestrated  with  other  faetors  to  aehieve  best 
value  within  the  proeurement  aetion. 
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III.  PROBLEMS,  CHALLENGES  AND  ANALYSIS 


A.  INTRODUCTION 

The  previous  chapters  addressed  the  guidance  surrounding  the  use  of  past 
performance  within  the  source  selection  process.  This  chapter  describes  challenges  and 
problems  encountered  by  users  of  the  process  and  provides  an  analysis  on  data  collected. 

B,  DATA  SOURCES 

The  following  data  sources  were  used  to  identify  the  challenges  and  problems 
encountered  with  past  performance  usage: 

•  Questionnaires  were  initially  sent  to  identify  primary  ways  past 
performance  was  used,  who  was  using  it,  and  when  it  was  being  used. 
(Appendix  B  contains  a  sample  of  questionnaire  used) 

•  Follow-up  telephone  interviews  were  conducted  to  assess  the  current 
compliance  with  the  guidance  on  past  performance.  Problems  and  current 
uses  of  past  performance  are  then  identified.  (Appendix  C  contains  sample 
of  interview  questions  asked) 

•  Typical  individuals  contacted: 

•  Individuals  engaged  in  past  performance  tracking 

•  Individuals  engaged  in  the  utilization  of  past  performance  data 

•  Individuals  using  and  maintaining  automated  databases  for  the 
collection  and  use  of  past  performance 

•  Typical  questions  included: 

•  What  automated  tools  are  currently  utilized? 

•  How  often  is  past  performance  used  as  a  source  selection  factor? 

•  How  are  you  implementing  the  current  Federal  Procurement  Policy 
guidance  on  the  use  of  past  performance? 

•  What  changes  are  necessary  to  fully  comply? 

•  What  do  you  think  the  role  of  past  performance  should  be? 

The  information  gathered  from  the  review  of  current  guidance  on  past 
performance  and  its  current  implementation  along  with  information  obtained  from 
questionnaires  and  interviews  as  well  as  organizational  experience  is  analyzed.  Problems 
with  current  past  performance  use  are  identified. 
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Data  collection  was  designed  to  eanvass  all  major  buying  commands  within  the 
Army  procurement  system.  The  questionnaires  were  focused  on  individuals  who  are 
administering  the  PPIMS  system,  eontraeting  personnel,  and  known  individuals  who  have 
used  past  performance  data  during  the  source  seleetion  process.  With  leads  generated 
from  the  questionnaires,  a  representative  population  was  interviewed  to  expand  on  the 
questionnaire  findings.  All  persons  contracted  were  either  contraet  speeialists,  contracting 
officers,  or  individuals  directly  in  the  area  of  data  past  performance  colleetion  and  use. 
The  table  below  outlines  the  demographies  of  the  questionnaire  and  interview  obtained. 


Table  3.1.  Questionnaire  and  Interview  Demographies 


AMCOM 

Questionnaires 

5 

Interviews 

4 

Contracting  Officers  and  Speeialist 

CECOM 

3 

2 

Contracting  Officers  and  Speeialist 

JMC 

3 

2 

Proeurement  Analyst 

Natick 

3 

2 

Contract  Specialist 

SBCCOM 

2 

0 

Contract  Specialist 

TACOM 

2 

2 

Contract  and  Procurement  Specialist 

HQAMC 

1 

0 

Procurement  Analyst 

The  information  above  represents  an  overall  52%  response  rate  to  the 
questionnaires  sent  out,  with  a  66%  response  rate  from  the  Army’s  primary  commands 
(AMCOM,  CECOM,  SBCCOM  and  TACOM).  The  data  reeeived  was  consistent 
between  eommands  and  was  considered  as  evidence  that  data  was  representative  of  the 
experiences  with  past  performanee  within  the  Army.  Interviews  were  eondueted  as 
follow-ups  to  questionnaires  to  gain  further  insights  into  the  use  of  past  performance  at 
eaeh  loeation. 

C.  QUESTIONNAIRE  AND  INTERVIEW  FINDINGS 
1,  Questionnaire  Results 

The  following  questions  were  asked  of  each  person  about  their  use  of  past 
performance.  The  percentages  represent  the  number  of  total  respondents  for  each 
category  or  area. 
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a.  What  Are  the  Primary  Ways  Past  Performance  Is  Used  in  Your 
Organization? 

%  of  respondents 


For  responsibility  determination  33% 

As  a  stand  alone  source  selection  factor  (along  with  price  and  other  factors)  89% 
As  a  sub  element  in  a  source  selection  67% 

Not  at  all  0% 


This  data  indicates  that  when  past  performance  is  used,  it  is  used  most 
often  as  a  stand-alone  factor  in  source  selection  process.  Past  performanee  was  also 
identified  as  a  sub  element  when  it  was  considered  along  with  other  factors  such  as 
technical  merit  factors. 

b.  Who  Determines  How  Past  Performance  Will  Be  Used  in  Your 
Organization? 

%  of  respondents 


Contracting  officer 

56% 

Policy  guidance  within  organization 

33% 

Contract  specialist 

0% 

Team  approach 

78% 

The  contraeting  officer  is  the  primary  foree  in  the  use  of  past  performance 
in  the  souree  selection  process.  This  individual  makes  the  ultimate  decision  if  it  will  be 
used  or  will  be  waived  as  a  non-factor  for  an  individual  procurement.  Although 
eontraeting  offieers  make  final  determination  on  its  use,  they  often  rely  on  input  from 
supporting  members  of  the  aequisition  workforee  in  making  the  deeision.  After  the 
contraeting  officers  has  decided  to  use  past  performance  data,  the  source  seleetion  official 
is  often  assisted  in  the  information  gathering  and  ranking  proeess  with  a  dedicated  team, 
often  called  a  Performance  Risk  Analysis  Group  (PRAG),  or  with  the  assistanee  of 
Preaward  monitors  within  the  commands.  If  the  source  selection  authority  is  someone 
other  than  the  contracting  officer,  he  or  she  may  have  the  final  say  on  use  of  past 
performanee  within  the  souree  selection. 
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c. 

What  Automated  Systems  Do  You  Use  to  Capture 
Performance  Data? 

(Respondents  could  check  any  that  apply) 

%  of  respondents 

PPIRS 

22% 

PPIMS 

78% 

CPARS 

22% 

DCMA  Databases 

56% 

CCSS  internal  data 

56% 

Proposal  data 

33% 

d.  Of  the  Past  Performance  Data  Used  How  Would  You  Rate  Its 


Effectiveness? 

Effectiveness 

Least 

Most 

1 

2 

3 

4 

5 

PPIRS 

50% 

50% 

0 

0 

0 

PPIMS 

43% 

14% 

14% 

29% 

0 

CPARS 

100% 

0 

0 

0 

0 

DCMA 

0 

40% 

40% 

20% 

0 

CCSS 

20% 

20% 

40% 

20% 

0 

Proposal  data 

0 

67% 

33% 

0 

0 

Within  this  data  set  the  respondents  were  asked  to  rank  the  effeetiveness 
of  the  data  gathered  using  eaeh  automated  system.  PPIMS  was  the  most  frequently  used, 
but  the  results  as  to  its  effeetiveness  were  mixed  with  57%  responding  that  it  was 
ineffeetive  and  43  %  responding  that  it  was  average  or  slightly  better  than  average.  The 
primary  reason  given  for  the  laek  of  effeetiveness  of  the  PPIMS  was  that  the  data 
provided  often  was  ineomplete  and  thus  did  not  aecurately  refleet  the  eontractor’s  total 
past  performanee  history.  Respondents  also  stated  that  the  data  revealed  no  new 
information  about  a  contraetor’s  past  performanee  history  as  the  data  entered  in  the 
system  was  often  entered  by  the  command  retrieving  the  data.  CPARS,  the  older  Army 
information  system,  was  rated  as  the  least  effective  among  all  respondents.  This 
perception  occurred  because  CPARS  was  replaced  by  PPIMS,  and,  as  a  result,  the  data 
contained  in  the  system  is  less  complete  than  the  data  found  in  PPIMS. 
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DCMA  databases  and  CCSS  internal  data  were  pereeived  as  the  most 
useful  data  with  60%  of  respondents  responding  that  both  systems  were  average  or  above 
in  usefulness.  The  DCMA  database  has  two  different  methods  of  traeking  a  contraetor’s 
past  performance.  Current  contracts  performance  is  measured  by  individual  CLIN  while 
completed  contracts  are  measured  at  the  contract  level.  This  information  is  only  retained 
for  a  two-year  period.  The  difference  in  the  rationale  for  the  way  the  information  is 
gathered  and  maintained  presents  several  problems  for  use  in  source  selection.  The  data 
that  reside  in  these  databases  often  have  not  been  shared  with  the  contractors  and  require 
disclosure  to  the  contractor  prior  to  use  in  the  selection  process.  Furthermore,  past 
performance  data  received  as  part  of  the  proposal  was  perceived  to  be  incomplete  and 
vague  with  a  distinct  bias  to  divulge  only  the  good  performance  of  the  contractor.  Despite 
the  flaws  in  the  DCMA  and  CCSS  databases,  both  systems  were  rated  favorably  because 
of  their  familiarity  to  the  users  and  the  easy  access  to  the  data  in  each  system. 

e.  What  Are  Some  of  the  Problems  that  You  Have  Experienced  with 
Past  Performance  Data? 

%  of  respondents 


Completeness  of  the  data  78% 

Labor  intensive  67% 

Determining  cause  for  poor  performance  33% 

Validation  of  data  with  contractors  56% 

Rating  performance  on  newly  formed  or  merged  companies  33% 

Relative  weight  past  performance  given  in  relation  with  other 

source  selection  factors  67% 


Overall,  there  is  a  high  level  of  dissatisfaction  with  the  completeness  of 
the  data  available  and,  as  a  result,  its  usefulness  in  the  source  selection  process. 

Compliance  in  filing  reports  when  required  and  completeness  of  data 
when  submitted  are  still  a  problem.  Data  is  often  incomplete  or  not  measured  in  a  like 
manner;  this  necessitates  a  large  amount  of  man-hours  to  be  expended  before  the  data  can 
be  effectively  used.  As  a  result,  large  amounts  of  time  were  required  by  the  source 
selection  team  to  correct  or  revalidate  the  data  provided  through  the  CCSS  system  or 
PPIMS. 


21 


The  information  collected  was  often  not  shared  with  the  contractor  prior  to 
being  input  into  the  databases.  As  a  result,  the  information  needed  to  be  shared  with 
contractor  prior  to  its  use  in  the  source  selection  process  in  accordance  with  FAR 
guidance.  Often  delays  in  delivery  are  mitigated  by  Government  delays  that  impact 
contractor  performance.  Without  validation  of  the  data  as  to  the  true  cause  for  the 
contractual  delays,  unjustified  conclusions  could  be  made  about  the  contractor’s 
performance. 

2,  Interview  Results 

As  a  follow  up  the  questionnaire  results,  12  individuals  were  interviewed  to 
acquire  more  in-depth  understanding  of  their  experiences  with  past  performance 
information.  This  phase  of  the  data  collection  process  proved  to  be  most  critical  to 
understanding  the  texture  of  the  problems  encountered  with  the  use  of  past  performance 
data.  The  following  are  the  primary  questions  asked  during  the  interviews  and  a 
summary  of  the  comments  provided. 

a.  What  Automated  Tools  Are  Currently  Utilized? 

Most  participants  interviewed  acknowledged  the  databases  asked  about  in 
the  questionnaire  phase.  They  also  voiced  their  dissatisfaction  with  problems 
encountered  with  their  use.  PPIMS  were  generally  recognized  as  the  primary  source  of 
past  performance  data  on  large  dollar  procurements.  The  problems  identified  with  the 
PPIMS  system  were  that  the  database  was  incomplete  as  all  contracts  that  should  have 
been  reported  in  PPIMS  was  not  and  those  that  had  some  data  were  often  not  up  to  date. 
When  asked  why  this  was  the  case,  the  answer  was  that  the  process  of  updating  the 
system  was  labor  intensive  and  often  was  a  low  priority  within  the  organization.  Many 
interviewed  had  real  time  insights  into  this  process,  as  they  were  often  the  same 
individuals  who  input  data  into  the  system  for  contracts  they  administered.  They  stated 
that  the  system  was  labor  intensive  with  respect  to  both  the  input  of  the  data  as  well  as 
retrieval  of  the  data  from  the  system. 

The  PPIMS  database  has  been  the  result  of  a  major  effort  to  try  to 
standardize  past  performance  data  collection  within  the  Army.  PPIMS  is  the  Army’s 
principal  tool  for  the  collection  and  maintenance  of  contractor  past  performance 
information  (PPI).  The  primary  purpose  of  PPIMS  is  to  provide  a  secure,  web-based. 
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information  system  providing  the  Army  a  user-friendly  tool  to  prepare  and  maintain 
eontractor  performance  reports.  It  became  operational  on  October  11,  1997,  with  actual 
on-line  reporting  capability  in  1998. 

The  primary  problem  has  been  that  the  data  required  for  input  into  the 
PPIMS  system  is  not  being  entered.  The  PPIMS  system  has  established  mandatory 
reporting  thresholds  of  $5M  dollars  for  systems  and  operations  support  contracts  and 
$1M  dollars  for  all  other  contracts.  The  levels  of  reporting  and  input  into  in  the  PPIMS 
system  have  been  significantly  lower  than  required.  As  a  result  the  data  is  often  not 
representative  of  a  contractor’s  actual  performance  on  these  contracts.  Consequently,  the 
source  selection  teams  must  then  look  to  other  sources  of  past  performance  data  for  their 
evaluation.  With  PPIMS  requiring  reporting  of  contract  performance  at  the  $IM  level, 
even  if  the  data  were  present,  one  could  argue  that  since  most  contracts  are  under  this 
reporting  threshold,  that  the  performance  history  is  incomplete  even  if  the  data  were 
present 

The  DCMA  database  was  also  identified  as  a  good  source  for  past 
performance  data.  The  database  often  was  more  detailed  than  other  systems  but  the 
duration  of  time  the  information  was  retained  in  the  database  was  identified  as  a  problem. 
The  data  often  had  to  be  shared  with  the  contractor  prior  to  its  use  as  this  was  not 
accomplished  previously.  The  DCMA  database  also  employs  an  alert  system  that  is  used 
to  identify  problems  with  contract  performance  to  the  buying  command.  This  system  was 
identified  as  a  useful  tool  to  determine  contract  performance. 

Each  buying  command  has  past  performance  data  residing  in  the  CCSS 
system  internal  to  their  commands.  This  data  is  limited  to  the  individual  buying 
command’s  past  performance  history  on  given  contracts.  This  system  provided 
significant  shipping  information  on  specific  contracts.  However,  the  system  does  not 
document  the  cause  for  the  delay  nor  provide  an  overall  assessment  of  a  contractor’s  total 
performance. 

AMCOM  developed  an  automated  program,  which  measured  a 
contractor’s  performance  with  the  command  on  an  ongoing  basis,  and  the  method  of 
measurement  was  computed  in  such  a  way  as  to  provide  a  uniform  methodology  and  two 
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metrics  associated  with  performance.  The  program  measured  contractor  performance 
using  the  units  delivered  over  the  last  two-year  period.  Two  measurements  were  captured 
on  this  performance  history.  The  contractor  was  measured  on  how  often  the  company 
shipped  units  early  or  on-time  versus  late  and  included  the  average  number  of  days  that 
deliveries  were  early  or  late.  This  information  was  made  available  to  the  contracting 
community  on  an  ongoing  basis,  which  limited  the  time  frames  required  for  review  of  the 
data.  This  approach  provided  texture  and  depth  to  the  analysis  of  a  contractor’s 
performance  by  measuring  not  only  on-time  performance,  but  also  the  degree  of 
performance  in  days  early,  on  time,  and  late.  This  approach  provided  a  measurement  at 
the  most  common  denominator  and,  as  a  result,  was  considered  by  the  buying  command 
as  the  fairest  method  of  measurement  possible.  The  problem  is  that  this  command  was 
the  only  one  to  track  to  this  level.  When  data  was  used  from  other  commands,  the  basis 
for  evaluation  was  often  different. 

b.  How  Often  Is  Past  Performance  Used  as  a  Source  Selection 
Factor? 

Respondents  indicated  that  past  performance  was  used  as  a  source 
selection  factor  only  a  limited  amount  of  the  time.  The  primary  reason  provided  was  that 
a  large  number  of  procurement  actions  are  sole  source  or  are  restricted  to  sources  by  a 
preferred  provider  list.  On  competitive  awards,  past  performance  was  primarily  used  only 
on  high  dollar  awards  well  over  $10M. 

Even  when  past  performance  was  used  in  the  source  selection,  it  carried 
minimal  weighting.  Its  lack  of  significant  weighting  in  the  source  selection  process 
resulted  in  the  final  rating  having  little  or  no  impact  in  the  source  selection. 

c.  How  Are  You  Implementing  the  Current  Federal  Procurement 
Policies  Guidance  on  the  Use  of  Past  Performance? 

In  most  cases,  the  contracting  officer  waived  the  use  of  past  performance 
on  the  basis  of  not  being  appropriate  in  the  current  contract  award.  The  primary  reason 
given  for  the  waivers  was  that  the  performance  history  of  potential  offers  was  already 
known  and  that  their  performance  was  not  divergent  enough  to  be  a  factor  in  the 
selection. 
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As  to  compliance  to  the  reporting  side  of  past  performance,  respondents 
felt  workload  and  higher  priorities  often  prevented  consistent  updates.  On  large 
procurements  where  the  requirer  was  the  program  offiee,  the  updating  of  the  PPIMS  data 
was  left  up  to  that  office.  This  updating  of  the  database,  however,  goes  beyond  the 
PPIMS  database.  No  matter  whieh  database  is  used,  the  aecurate  and  timely  updating  of 
the  data  is  paramount.  If  performance  is  not  maintained  in  the  other  databases  sueh  as 
CCSS  and  DCMA’s  MOCAS  database,  these  data  sources  also  become  less  useful  when 
considering  past  performanee. 

The  guidanee  requires  past  performance  to  be  evaluated  in  all  eompetitive 
negotiated  contraets  over  $100,000.  It  also  allows  for  a  waiver  if  it  is  determined  by  the 
eontracting  offieer  that  past  performance  inclusion  as  a  selection  factor  is  not  appropriate. 
Based  on  the  number  of  times  past  performance  is  used  as  a  souree  selection  factor,  this 
waiver  is  used  in  the  large  majority  of  procurements.  The  question  then  arises  as  to  why 
waivers  are  pursued?  The  answer  lies  in  two  areas.  The  first  is  the  eumbersome  nature  of 
past  performance  data  when  used.  The  second  reason  is  that  past  performance  is  not 
perceived  as  a  good  predictive  indieator  of  future  performance,  and  past  performance  in 
the  first  place  is  seen  to  have  limited  value. 

Past  performanee  should  always  be  a  primary  eonsideration  in  the 
responsibility  determination  that  a  contracting  officer  makes  prior  to  any  award. 
However,  when  it  is  used  as  a  selection  factor  along  with  price,  its  importance  in  the 
award  decision  is  raised  to  a  higher  level.  The  importance  of  past  performance  can  be 
diminished,  however,  by  how  much  weight  it  is  afforded  in  the  source  selection  process. 
Furthermore,  the  collection  and  validation  of  past  performance  data  ean  be  both  a 
resource-intensive  and  time-consuming  process;  therefore,  when  past  performance  is 
determined  to  be  appropriate  for  use,  its  weighting  in  the  souree  seleetion  proeess  should 
be  significant  enough  to  be  able  to  influence  the  souree  selection  outcome.  Respondents 
interviewed  felt  that  unless  a  weight  of  at  least  25%  is  afforded  an  element,  it  beeomes 
diffieult  for  the  rating  to  have  any  impact  in  the  selection  process. 
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d.  What  Changes  Are  Necessary  to  Fully  Comply  with  Guidance  on 
Collection  and  Use  of  Past  Performance  Information? 

Currently  there  is  a  low  priority  given  to  the  maintenanee  of  the  past 
performanee  databases.  Due  to  manpower  reduetions,  individuals  historieally  foeused  on 
this  area  no  longer  reside  in  the  eommands;  eonsequently  this  responsibility  has  beeome 
another  task  as  assigned  to  the  eontraet  speeialist.  Beeause  of  the  eurrent  environment 
emphasizing  making  awards  faster,  less  emphasis  is  put  on  eontraet  administration. 
Several  individuals  interviewed  stated  that  a  eentrally  eontrolled,  group  foeused  on  past 
performanee  would  help.  The  guidanee  of  rating  eontraetors  with  no  performanee  history 
as  neutral  also  was  mentioned  as  problematie.  It  was  felt  by  some  respondents  that  a  laek 
of  past  performanee  was  representative  of  risk  on  performance  of  the  current 
requirements. 

Consistency  of  past  performance  data  was  a  major  concern  by  all 
individuals  contacted.  The  PPIMS  database  has  been  a  major  effort  to  try  to  standardize 
past  performance  data  collection  within  the  Army.  PPIMS  is  the  Army’s  principal  tool 
for  the  collection  and  maintenance  of  contractor  past  performance  information  (PPI). 

The  primary  problem  has  been  that  the  data  required  for  input  into  the 
PPIMS  system  is  not  being  entered.  The  PPIMS  system  has  established  mandatory 
reporting  thresholds  of  $5M  dollars  for  systems  and  operations  support  contracts  and 
$1M  dollars  for  all  other  contracts.  The  levels  of  reporting  and  input  into  in  the  PPIMS 
system  have  been  significantly  lower  than  required.  As  a  result,  the  data  is  often  not 
representative  of  a  contractor’s  actual  performance  on  these  contracts.  Consequently,  the 
source  selection  teams  must  then  look  to  other  sources  of  past  performance  data  for  their 
evaluation.  With  PPIMS  requiring  reporting  of  contract  performance  at  the  $IM  level 
even  if  the  data  was  present  one  could  argue  that  since  most  contracts  are  under  this 
reporting  threshold,  that  the  performance  history  is  incomplete  even  if  the  data  were 
present.  [Ref.  1  :Paragraph  a] 

The  measurement  of  a  contractor’s  effectiveness  in  overcoming  problems 
associated  with  contract  performance  should  also  be  considered  when  a  trend  could  be 
established.  DoD  currently  does  not  have  a  system  that  automates  the  collection  of  this 
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kind  of  past  performance  data  in  an  effective  manner  and  only  requires  assessments  by 
administrators  on  issues  associated  with  trends  and  resolution  of  gray  areas  that 
automated  data-collection  cannot  achieve. 


The  FAR  states  that  the  length  of  time  that  past  performance  data  are  to  be 
retained  should  not  be  exceed  three  years  after  completion  of  contract  performance.  In 
most  cases  this  makes  sense  because  past  performance  should  be  recent  enough  to  be 
predictive.  There  are  cases,  however,  like  major  weapon  system  procurements,  that  data 
up  to  five  years  or  more  has  some  relevancy.  An  example  of  this  is  when  a  major 
weapon  system  is  being  procured  and  the  relevant  procurement  history  for  a  similar 
system  is  longer  than  three  years. 

Contractors  have  complained  about  the  way  that  the  three-year  duration  is 
interpreted,  since  completion  of  a  contract  is  often  not  recorded  with  shipments.  Rather, 
because  the  contract  has  not  been  administratively  closed  out,  the  performance  on  the 
contract  is  included  even  though  shipment  performance  was  completed  more  than  three- 
years  previously.  There  is  also  debate  whether  all  performance  on  the  contract  may  be 
utilized  or  just  the  portion  of  performance  that  is  within  the  last  three  years. 

Another  concern  or  potential  problem  with  past  performance  is  the 
assessment  of  the  contractor  with  no  performance  history,  or  the  assessment  of  newly 
formed  companies.  Current  guidance  indicates  that  these  contractors  should  be  rated  as 
unknown,  having  no  positive  or  negative  evaluative  significance.  This  approach  ignores 
that  a  lack  of  performance  by  a  company  in  itself  indicates  an  unknown  risk  as  to  their 
ability  to  perform  on  the  current  proposed  contract. 

A  similar  problem  occurs  when  contractors  team  to  bid  on  a  proposal.  In 
this  case  a  determination  is  made  to  determine  which  contractor  is  providing  what  efforts 
and  then  applying  their  individual  past  performance  histories  accordingly.  This  method 
may  also  be  applied  if  there  is  a  prime,  who  has  a  subcontractor  that  is  being  relied  on 
heavily. 
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e.  What  Do  You  Think  the  Role  of  Past  Performance  Should  Be? 

The  overwhelming  response  was  that  past  performanee  should  only  be 
eonsidered  as  a  souree  seleetion  factor  when  conditions  warranted  its  use.  Past 
performance  should  primarily  be  considered  when  poor  performing  contractors  are 
anticipated  to  be  bidding,  and  when  nonresponsibity  determination  cannot  be  justified  but 
performance  is  not  as  good  as  the  command  would  like.  Often  nonresponsibility 
determinations  made  by  the  contracting  officer  are  overturned  by  the  small  business 
office  in  the  form  of  certificate  of  competency  (CoC).  By  making  past  performance  a 
source  selection  factor  only  on  responsible  contractors,  the  ability  of  small  business  to 
issue  a  CoC  is  negated  because  a  small  business  non-selection  was  based  on  best  value 
criteria  and  not  solely  on  a  determination  of  responsibility.  An  SBA  certificate  of 
competency  becomes  moot  in  this  case,  requiring  that  past  performance  of  small  business 
competitors  be  scrutinized,  the  same  as  any  other  competitor  Another  reason  given  for  the 
use  of  past  performance  as  a  source  selection  factor  is  when  delivery  of  the  item  is  critical 
to  the  readiness  of  the  command.  Most  often  when  this  occurs,  urgency  is  used  as 
justification  for  sole  source,  thus  avoiding  competition  in  the  first  place. 

In  general,  past  performance  as  a  source  selection  factor  was  not  well 
received  by  most  interviewed.  The  list  below  outlines  other  comments  provided  about 
the  use  of  past  performance  as  a  source  selection  factor. 

•  “Almost  never  prevents  award  to  lowest  bidder” 

•  “There  are  always  mitigating  circumstances  for  poor  performance” 

•  “Make  it  go  away” 

•  “Can  provide  significant  insights  to  a  contractor’s  capabilities” 

•  “Rating  are  sugar  coated  or  artificially  raised” 

•  “Should  not  be  so  labor  intensive  to  achieve  results” 

•  “Legal  community  sometimes  overtly  conservative” 

•  “Only  a  tool” 

Several  respondents  stated  that  the  legal  support  in  the  source  selection 
process  often  took  a  conservative  approach  to  the  grading  of  contractors,  which  often  led 
to  more  conservation  or  neutral  ratings.  The  legal  communities  in  one  command  even 
reviewed  all  the  write-ups  on  each  contractor  and  were  overbearing  in  requiring  changes 
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and  recommendations,  which  team  members  did  not  agree  with.  Legal  counsel  is 
important  part  of  the  source  selection  process  but  should  not  unduly  influence  the 
judgment  or  conclusions  of  the  raters. 

D,  OTHER  RELATED  STUDIES 

The  American  Bar  Association’s  Public  Contract  Law  Section  completed  an 
extensive  study  on  the  Government’s  current  practice  of  past  performance  evaluations  in 
May  of  2003.  The  findings  of  this  study  closely  parallel  the  information  gathered 
through  my  questionnaires  and  interviews.  Their  study  had  over  1315  participants  made 
up  of  Government  contracts  management,  corporate  contracts  management,  consultants. 
Government  and  corporate  attorneys,  private  law  firms,  and  others  that  responded  to  the 
study.  The  American  Bar  Association  study  identified  the  following  information  with 
regards  to  past  performance; 

Some  of  the  problems  areas  identified  in  the  study  were  the  consistency  in  the 
reporting  requirements  as  well  as  the  overall  grading  process.  The  use  of  past 
performance  was  only  considered  moderately  effective  in  predicting  a  contractor’s  future 
performance.  Only  28%  of  respondents  thought  that  it  was  an  accurate  predictor  of  a 
contractor’s  future  performance.  [Ref.  22:Question  3] 

All  respondents  were  asked  the  following  questions: 

1.  The  Greatest  Hurdles  to  the  Government-Wide  Implementation  of  the 
Use  of  Past  Performance  Are  (Choose  As  Many  As  Are  Appropriate): 

•  Inconsistency  in  reporting/grading  65%  of  all  respondents 

•  Inconsistency  in  applying  data  and/or  47.3%  of  all  respondents 

using  reports 

•  Inability  to  distinguish  levels  of  39.2%  of  all  respondents 

performance 

•  Inability  to  adequately  comment  or  3 1 . 1  %  of  all  respondents 
challenge 

•  Evaluator’s  fear  of  reprisal  for  negative  18.4%  of  all  respondents 
reports 

•  Gives  the  Contracting  Officer  too  much  15.3%  of  all  respondents 
discretion 

•  Fundamentally  a  bad  idea  4%  of  all  respondents  [Ref 

22:  Question  2] 
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This  data  once  again  highlights  the  problems  with  inconsistency  in  grading  and 
reporting  past  performance  information.  Often  levels  of  distinction  between  levels  are 
unclear,  and  evaluators  are  reluctant  to  grade  down;  and,  as  a  result,  they  artificially  level 
the  playing  field. 

2.  Comments  by  Government  Agencies 

The  use  of  past  performance  as  a  significant  evaluation  factor  has: 

•  Helped  to  eliminate  poor  performing  contraetors  (32. 1%  of  respondents). 

•  Produced  improvement  in  contraetor  performance  (27.9%  of  respondents). 

•  Increased  my  workload  (27.2%  of  respondents). 

•  Enhaneed  my  ability  to  select  superior  contractors  (26.3%  of  respondents). 

•  Failed  to  alter  eontractor  performanee  (14.7%  of  respondents). 

•  Not  worth  the  effort  ( 1 3.9%  of  respondents). 

•  Made  source  selection  more  burdensome  (13.9%  of  respondents).  [Ref 
22:  Question  22] 

The  data  above  indicate  that  that  only  one  third  of  the  respondents  felt  that  past 
performanee  eliminated  poor  performing  contraetors.  The  use  of  past  performanee  as  a 
source  seleetion  factor  has  had  limited  sueeess  improving  the  performanee  of  contractors 
as  only  28%  of  respondents  stated  that  they  felt  the  use  of  past  performance  as  a  source 
selection  factor  produced  improvement  in  contractor  performance.  The  use  of  past 
performance  also  enhaneed  the  contracting  officer  ability  to  select  superior  eontractors 
among  only  26%  of  respondents.  This  data  indicates  that  while  the  use  of  past 
performance  data  in  the  source  selection  process  increases  the  workload  of  the 
contracting  officials  and  makes  the  souree  seleetion  proeess  more  burdensome  the 
benefits  my  not  be  significant  enough  to  warrant  its  use. 

(L  Do  You  Use  PPIRS? 

No  54%  of  respondents 

Yes  46%  of  respondents  [Ref  22:Question  26] 

This  indicates  that  all  respondents  do  not  use  the  PPIRS  system  and, 
therefore,  rely  on  other  forms  of  past  performance  data  in  making  their  deeisions.  The 
PPIRS  proeess  is  simply  a  web  info  site,  whieh  provides  information  to  aeeess  the 
individual  past  performanee  databases. 


30 


b.  Do  Contractors  ALWAYS  Have  an  Opportunity  to  Comment 
Upon  a  Negative  Report? 

No  86%  of  respondents 

Yes  31%  of  respondents  [Ref  22:Question  28] 

This  response  highlights  a  major  concern  within  the  contracting  and 
contractor  community:  that  despite  guidance  to  the  contrary,  past  performance,  in 
particular  negative  past  performance,  is  not  reviewed  and  validated  with  the  contractor 
prior  to  being  used  in  the  source  selection  process. 

3.  Contractor  Comments 

•  Failure  to  have  and  use  automated  systems  required  Government 

customers  to  manually  answer  questionnaires  over  and  over  again. 

•  PPl  used  as  club  or  weapon  by  the  Government. 

•  Inconsistency  and  subjectivity  are  a  problem. 

•  Contractors  are  afraid  to  exercise  rights. 

•  Past  performance  only  works  to  distinguish  top  performers. 

•  Small  performers— small  business  also  a  problem. 

•  Agencies  still  not  complying  with  FAR  requirements. 

•  Need  to  standardize. 

•  Failure  to  use  automated  system  creates  burdens  in  responding  to 

questionnaires. 

•  Afraid  to  give  anything  but  average  ratings—  inflation  of  scores. 

•  Need  more  training,  guidance,  and  resources. 

•  Subjective  and  inconsistent. 

•  Does  not  provide  useful  management  tool. 

•  Does  help  select  better  performing  contractors. 

•  CPARS  cumbersome  and  confusing. 

•  Problem  with  evaluating  offers  with  no  past  performance  [Ref 

22:Question  32] 

These  comments  illustrate  many  of  the  concerns  with  the  use  of  past  performance 
data.  Failure  to  have  a  reliable  automated  database  on  past  performance  requires  manual 
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labor  to  accomplish  reviews.  These  efforts  are  often  repeated  again  and  again  for  eaeh 
source  selection.  Past  performance  ratings  are  often  considered  more  subjective  than 
objeetive  and  ean  vary  between  souree  selections. 

The  level  of  consisteney  between  and  among  source  seleetion  was  a  eoneem 
brought  out  in  both  the  interviews  with  the  command  and  in  the  ABA  study.  The  degree 
of  consisteney  is  often  a  function  of  the  individuals  performing  the  evaluation.  The 
Army  reeommends  the  formation  of  a  Performance  Risk  Analysis  Group  (PRAG)  for 
large  procurements  over  $40  million.  The  use  of  a  PRAG  for  the  execution  of  the  past 
performanee  evaluations  greatly  helps  to  maintain  the  eonsisteney  of  the  evaluation  and 
application  of  the  risk  ratings.  A  dedicated  group  can  apply  like  logic  to  the  ratings  and 
eonsisteney  against  the  souree  selection  plan.  Most  interviewed,  however,  felt  that  this 
dedicated  group  was  only  used  on  large  proeurements.  On  smaller  proeurements  eontraet 
specialists  are  left  on  their  own  to  perform  the  analysis.  As  a  result  of  downsizing  over 
the  past  five  years,  eaeh  eommand  has  lost  the  core  group  that  specialized  in  past 
performanee  traeking  and  contractor  evaluation  during  the  souree  seleetion  proeess.  This 
loss  has  required  the  eontraet  specialist  to  gather  the  necessary  information  and  apply  the 
souree  selection  plan  without  the  help  of  individuals  who  have  extensive  experience  with 
past  performanee  data  and  its  applieation  to  the  source  selection  process. 

4,  Other  Observations 

One  of  the  primary  dissatisfaetions  voiced  about  the  measurement  and  use  of  past 
performance  currently  is  that  it  does  not  address  the  eore  question  of  every  requirer, 
which  is  “What  is  the  risk  to  my  program  or  eontraet?”  Past  performance  measurement 
currently  derives  a  risk  rating  that  focuses  on  the  contraetor’s  relevant  and  recent 
eontractual  performanee.  This  performance  has  varying  levels  of  direct  correlation  to  the 
current  contract.  The  greater  the  correlation  between  the  eontraetor’s  past  performanee 
and  the  requirements  of  the  eurrent  eontraet,  the  higher  the  confidenee  that  this 
performanee  history  will  continue  into  the  future.  The  problem  comes  when  this  current 
contract’s  requirements  vary  from  a  contraetor’s  past  performanee.  Factors  such  as 
production  rates,  vendor  relationships,  new  partnerships  or  team  relationships,  change  in 
capaeity,  and  size  of  the  procurement  can  directly  impact  the  risk  of  performance  by  a 
eontraetor  regardless  of  its  performanee  history. 


32 


Currently,  these  elements  are  often  addressed  in  preaward  survey,  whieh  is 
primarily  used  to  address  the  responsibility  determination.  These  surveys  provide  the 
buying  eommand  with  information  that  resulted  in  an  award/  no  award  determination. 
They  also  do  not  define  levels  of  risk  assoeiated  with  eaeh  element.  As  stated  previously 
best  value  procurements  are  designed  to  only  consider  responsible  contractors  for  the 
tradeoff  of  other  factors  such  as  past  performance  during  the  source  selection  process.  As 
a  result  past  performance  is  used  from  an  historical  perspective  and  then  applied  to  future 
requirements  for  source  selection. 

One  way  to  more  effectively  deal  with  this  problem  is  to  back  away  from  making 
past  performance  a  stand  alone  factor.  Past  performance  as  an  indicator  of  future 
performance  should  be  utilized  in  conjunction  with  other  factors  that  take  into  account 
the  contractor’s  current  capabilities  with  regards  to  an  upcoming  procurement.  On  larger 
procurements  past  performance  should  be  considered  along  with  other  factors  to  assess  a 
contractor’s  total  risk  in  performing  on  the  current  procurement.  In  this  approach, 
contractors  who  are  considered  responsible  are  evaluated  by  a  small  group  of  individuals 
who  provide  an  assessment  of  risk  levels  associated  with  the  contractor’s  delivery 
performance  as  well  as  other  performance  factors  such  as  facilities,  financial,  personnel, 
subcontract  control  and  quality  history.  These  assessments  primarily  focus  on  each 
contractor's  capability  risks  in  the  performance  of  an  upcoming  requirement  based  on 
current  conditions  and  the  manner  in  which  the  contractor  performed  in  these  areas  in  the 
past. 

This  approach  provides  the  acquisition  community  and  the  source  selection 

authority  with  an  estimate  of  the  contractor’s  ability  to  perform  on  the  current 

requirement,  not  just  past  performance.  This  information,  resulting  from  a  capability  risk 

assessment  can  then  be  utilized  in  conjunction  with  price  and  other  factors  in  the 

selection  of  the  contractor  who  provides  the  best  value  to  the  Government.  This  approach 

allows  for  the  inclusion  of  past  performance  history  in  the  source  selection  process  but 

also  takes  in  consideration  the  risk  that  contractors  bring  to  the  current  acquisition.  The 

use  of  this  approach  provides  a  more  useful  and  actionable  metric  by  quantifying  the  risk 

associated  with  each  contractor.  For  this  approach  to  be  most  effective  a  team  of 

functional  experts  should  go  onsite  to  the  contractor’s  place  of  performance  to  conduct 
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the  assessment.  The  team  should  interfaee  with  the  servieing  Defense  Contraet 
Management  Ageney  (DCMA)  offiee  to  validate  previously  eollected  performance 
history  and  draw  on  their  expert  knowledge  of  the  contractor.  The  core  team  should  be 
maintained  in  assessing  each  contractor’s  risks.  This  team  might  comprise  specialists 
inside  the  command  and/or  outside  support  on  large  procurements.  The  important 
elements  of  this  group  are  that  they  function  as  a  team  throughout  the  total  source 
selection  process.  This  would  help  ensure  continuity  of  assessment,  which  is  instrumental 
in  the  integrity  of  the  assessment.  This  approach  is  more  expedient  than  the  current  past 
performance  approach,  providing  the  source  selection  authority  with  a  risk  assessment  of 
not  just  a  contractor’s  performance  on  previous  contracts  but,  rather,  an  assessment  of  the 
relationship  between  past  performance  and  the  current  requirements  of  the  program.  An 
approach  that  couples  past  performance  information  with  an  assessment  of  risks  for 
performing  on  the  current  requirement  would  demonstrate  an  effective  use  of  past 
performance  information.  Risk  information  would  allow  the  source  selection  authority  to 
make  a  sounder  and  more  defensible  decision  for  trading  off  the  risk  with  other  factors  in 
the  source  selection  process.  The  table  below  compares  the  current  process  of  past 
performance  and  the  proposed  use  of  past  performance  within  a  contractor  capability  risk 
assessment  of  the  contractor. 


Table  3.2  Current  Past  Performance  Process  Compared  to  Contractor  Capability 

Risk  Assessment  Process 


Current  Process 

Contractor  Risk  Assessment  Process 

•  Focuses  on  each  contractor’s  past 
performance  history 

•  Assessments  often  more  subjective 
than  objective 

•  Past  performance  often  incomplete 
and  neutral  in  nature 

•  Often  time  consuming  and  labor 

•  Intensive 

•  Validation  of  past  performance 

•  Information  often  difficult  through 
questionnaires  and  interviews 

•  Uses  past  performance  data  as  baseline 
but  also  assesses  current  capabilities 
along  with  past  performance  data 

•  Validation  of  past  performance  can  be 
accomplished  during  onsite  visit 

•  Differences  in  current  environment  can 

•  Be  accounted  for  and  risk  properly 
assessed 
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The  use  of  past  performanee  as  an  evaluation  factor  in  the  source  selection 
process  should  be  used  primarily  when  there  is  a  historical  basis  of  performance  either 
with  previous  producers  or  with  similar  items  produced.  When  items  being  procured 
affected  the  impacts  readiness  of  a  weapon  system  because  of  the  Government’s  supply 
position  and/or  past  history  of  difficulty  of  obtaining  the  parts,  then  past  performance  in 
the  selection  process  becomes  increasingly  important.  Past  performance  should  be 
considered  in  the  responsibility  determination  of  every  contract,  even  if  it  is  not  a 
separate  factor  in  the  award  process. 

The  preoccupation  with  reducing  cycle  time  and  therefore  reducing  PALT  has 
influenced  the  use  of  past  performance  in  the  source  selection  process.  This  push  to 
award  contracts  in  less  time  has  resulted  in  added  pressure  not  to  consider  past 
performance  because  it  adds  to  the  time  required  to  make  the  award.  The  FAR  requires 
that  past  performance  information  collected  on  a  contractor  be  shared  with  the  contractor. 
This  will  allow  for  the  contractor  to  validate  the  accuracy  of  the  data  and  resolve  any 
differences  prior  to  its  use.  The  FAR  allows  30  days  for  past  performance  review  of  data 
by  the  contractor.  Often  this  process  can  take  longer  than  30  days  when  cause  for  the 
delay  is  mitigated  by  the  Government’s  actions  during  the  performance  of  the  contract. 
The  knowledge  that  the  use  of  past  performance  in  the  source  selection  process  could  add 
up  to  several  months  in  the  source  selection  process  is  a  strong  deterrent  to  its  use  in 
today’s  environment.  A  process  of  collecting  and  validating  data  prior  to  its  use  can  help 
prevent  this  additional  cycle  time.  PPIMS  does  have  a  process  to  validate  data  prior  to  its 
use,  but  this  data  is  often  incomplete. 

A  historical  problem  with  the  measurement  of  performance  is  the  method  of 
measurement.  If  contract  performance  is  measured  on  a  contract  basis,  the  results  can  be 
different  than  if  measured  in  on  a  contract  line  item  nomenclature  (CLIN)  basis.  Since 
each  contract  can  have  several  CLINs  outlining  the  deliverables  and  the  contract 
schedule,  the  number  of  data  points  expand  if  contract  performance  is  measured  using 
CLINs.  The  way  contracts  are  written  as  to  the  number  of  CLINs  used  can,  therefore, 
significantly  affect  the  results  of  any  measurement. 
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Several  respondents  from  AMCOM  provided  insights  into  their  experiences  with 
past  performance  usage  in  the  source  selection  process  while  with  the  Army  Troop  and 
Aviation  Command  (ATCOM).  The  table  below  represents  data  collected  by  ATCOM 
during  the  period  1990  through  1996.  The  data  illustrates  several  things  about  the  use  of 
past  performance  including  reluctance  to  use  it  in  the  source  selection  process. 

Table  3.3.  Past  Performance  History  ATCOM  (From;  ATCOM,  Production 

Management  Division) 

Summary  of  use  of  past  performance  within  the  source  selection  process 


Yea  r 

PWDs 

Reviewed 

Nominated 
for  inclusion 

Awarded 

w/  PPI  as  low  bidder 
An  evaluation 

Factor 

Other  than 

To  low  bidder 

1990 

20 

3 

2 

1 

1 

1991 

36 

9 

1 

1 

0 

1992 

463 

245 

15 

0 

15 

1993 

92 

31 

17 

3 

14 

1994 

136 

27 

2 

1 

1 

1995 

935 

37 

1 

1 

0 

1996 

925 

15 

1 

1 

1 

Production  and  Quality  Specialists  reviewed  individual  Procurement  Work 
Directives  (PWDs)  to  determine  if  an  item  being  procured  had  any  history  of  poor  quality 
or  late  delivery.  As  a  result  of  this  information,  the  contracting  officer  made  the  final 
determination  whether  or  not  past  performance  would  be  an  evaluation  factor  in  the 
solicitation.  Even  when  used,  the  weighting  proportioned  to  this  factor  was  under  the 
25%  weighting  recommended  by  the  technical  specialist.  As  a  result,  over  a  seven-year 
period  during  which  several  thousand  PWDs  were  considered  and  several  hundred 
nominated,  only  eight  were  awarded  to  other  than  the  low  bidder  in  the  final  analysis. 
These  results  even  further  bring  into  question  the  value  of  past  performance  data  since  in 
the  long  run  the  data  has  little  or  no  effect  the  contract  award. 

Some  respondents  as  well  as  many  within  DoD  also  believe  that  delivery 
performance  should  only  be  measured  against  the  original  schedule  in  the  contract.  The 
rationale  for  this  belief  is  that  the  original  schedule  reflects  the  expected  delivery  of  the 
item  by  the  Government.  Failure  to  meet  the  original  contract  schedule  represents  a 
breach  to  the  schedule  that  was  initially  required  by  the  Government.  The  primary 
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exception  to  this  would  be  if  some  Government  action  caused  the  delay  in  the  delivery. 
Measurement  to  original  delivery  schedules  is  made  impossible  in  an  automated  way 
since  this  information  is  not  routinely  maintained  in  the  databases  such  as  CCSS  and 
DCMA. 

Government-caused  delays  are  often  difficult  to  assess.  Often  gray  areas  such  as 
an  unclear  technical  data  package  (TDP)  being  provided  by  the  Government,  late 
Government  Furnished  Material  (GFM),  untimely  Government  responses  to  contractor’s 
questions,  late  approvals  of  first  article  or  product  verification  tests  make  it  difficult  to 
assess  the  contractor’s  responsibility  in  the  resulting  delays.  As  a  result,  data  is  often 
neutralized  or  thrown  out  during  the  source  selection  evaluation. 

E,  SUMMARY 

The  surveys  and  follow-on  interviews  indicated  several  overriding  patterns  in  the 
use  of  past  performance.  They  are  summarized  as  follows: 

•  Past  performance  data  is  not  used  as  frequently  as  the  policy  guidance 
directs. 

•  Past  performance  data  still  is  lacking  consistency  in  the  information 
systems  currently  trying  to  capture  the  data. 

Past  performances  usefulness  in  the  source  selection  process  is  perceived  as 
limited.  Many  problems  still  exist  with  both  the  gathering  of  past  performance  data  and  in 
its  use.  The  process  is  still  handicapped  by  the  lack  of  effective  automated  tools  that,  in 
the  absence  of  sufficient  manpower,  can  be  used  to  effectively  gather  and  use  past 
performance  data.  Its  use  is  also  inhibited  by  a  lack  of  dedicated  individuals  in  the 
buying  commands  to  assist  in  both  the  gathering  and  input  of  the  data  and  its  eventual  use 
in  the  source  selection  process. 

Past  performance  data  still  suffers  from  many  of  the  same  problems  that  have 
limited  its  use  in  the  past.  With  its  elevation  to  a  source  selection  factor,  these  problems 
are  amplified.  The  time  consumed  in  gathering  and  maintaining  the  data  is  made  more 
burdensome  by  the  reduced  workforce.  Lack  of  automated  systems  that  could  generate 
the  data  prevents  leverage  that  would  offset  the  reduction  of  the  workforce. 
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IV.  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  INTRODUCTION 

This  chapter  summarizes  the  research  presented  in  this  thesis  by  reviewing  the 
primary  and  the  secondary  research  questions  and  stating  the  eonclusions  that  result  from 
analysis  of  the  data  gathered.  The  ehapter  eoneludes  with  the  researeher’s  reeommended 
areas  for  further  study  and  analysis. 

Data  was  eollected  through  researeh  of  past  performance  information, 
questionnaires,  interviews  with  individuals  engaged  in  past  performance  data  gathering 
and  use  in  the  souree  seleetion  proeess  as  well  as  eollaborating  past  performanee  studies. 
The  data  indieated  that  eontraet  speeialists  still  eneounter  problems  with  their  use  of  past 
performanee  data,  whieh  resulted  in  various  levels  of  dissatisfaetion  with  its  use  in  the 
source  seleetion  proeess. 

Many  of  the  same  problems  identified  in  the  “Final  Report  for  the  Contraetor  Past 
Performanee  System  Evaluation  Study”  to  the  Deputy  Under  Seeretary  of  Defense 
(Aequisition  Reform)  in  June  of  1996  still  exist  today.  The  report  identified  a  number  of 
barriers  to  the  widespread  adoption  of  past  performanee  as  a  major  seleetion  faetor  in  the 
souree  seleetion  proeess.  The  areas  identified  were  as  follows: 

•  Weak  and  inaecurate  quality  and  delivery  data  proeesses 

•  Laek  of  tools  to  eolleet  aeeurate  data 

•  Time  to  validate  performanee  data 

•  Impact  on  the  acquisition  streamlining  efforts  to  redueed  procurement 
administrative  lead  times  (PALT) 

•  Produetivity  impaets 

•  Administrative  burden 

•  Laek  of  experienee  with  subjective  decision  making 

•  Lack  of  tools  to  collect  accurate  data 

•  Low  bidder  mind  set 

•  Risk  avoidanee  culture  [Ref  5:p.  38] 
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B  CONCLUSIONS  AND  SUMMARY 

Two  of  the  research  questions  pertained  to  the  guidance  surrounding  past 
performance  and  its  overall  compliance  within  the  Army  community:  “What  guidance 
currently  exists  on  past  performance?”  and  “What  is  the  overall  compliance  with  past 
performance  guidance?” 

Current  guidance  for  the  use  of  past  performance  has  recognized  many  of  these 
concerns  identified  in  the  previous  chapter.  OFFP  Policy  Letter  92-5  established  the 
requirements  for  collection  and  use  of  PPI  in  the  source  selection  process.  The  threshold 
at  which  past  performance  data  is  to  be  collected  and  for  its  use  has  been  revised.  A 
waiver  for  the  use  of  past  performance  data  as  a  factor  in  a  best  value  procurement  is 
allowed.  Past  performance  does  not  have  to  be  used  when  the  contracting  officer  can 
make  a  determination  that  there  is  no  significant  difference  in  the  past  performance  of  the 
contractors  that  will  bid  on  the  solicitation.  This  occurs  most  often  when  the 
procurements  are  restricted  to  other  than  full  and  open  competition  and  the  contracting 
officer  knows  the  contractors  that  are  eligible  to  bid. 

Another  research  question  asked  “What  does  past  performance  actually 
represent?”  Past  performance  has  not  had  the  impact  that  many  envisioned  when  it  was 
introduced  as  a  factor  along  with  price  in  the  best  value  process.  Many  problems  still 
exist  with  its  collection  and  use.  Despite  these  problems,  a  contractor’s  past  performance 
should  and  does  have  a  bearing  on  its  ability  to  perform  successfully  on  future 
requirements.  The  challenge  is  to  find  the  best  fit  for  its  use  in  the  source  selection 
process. 

This  research  indicates  that  the  use  of  past  performance  in  the  source  selection 
process  suffers  from  many  of  the  same  problems  that  it  has  since  its  inception  as  a  factor 
other  than  price  in  the  award  process.  The  use  of  past  performance  represents  a 
contractor’s  success  in  meeting  previous  contractual  obligations.  The  measurement  of 
contractor’s  meeting  those  obligations  is  made  difficult  by  data  collection  problems  and 
other  consideration  that  often  effect  performance  such  as:  government  caused  delays, 
changes  in  contract  scope,  vendor  or  manufacturing  problems.  Research  date  indicates 
that  the  use  of  past  performance  has  not  always  generated  the  desired  impacts  in  the 
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source  selection  process.  Many  individuals  feel  that  the  effort  required  to  use  past 
performance  data  and  the  inability  to  factor  other  considerations  that  affect  past 
performance  offset  any  benefits  derived  from  using  this  data. 

C.  HOW  IS  PAST  PERFORMANCE  USED  TODAY? 

Today  past  performance  is  primarily  used  in  assessing  the  responsibility  of  a 
given  contractor.  When  past  performance  is  used  as  another  factor  along  with  price  to 
determine  best  value,  it  takes  on  a  more  divergent  role  and  its  measurement  is  more  likely 
to  come  into  question.  The  role  of  past  performance  in  many  respects  is  clearer  when 
used  as  a  pass/fail  indicator,  as  in  a  responsibility  determination.  If  a  contractor  has 
demonstrated  poor  performance  in  the  past,  then  its  responsibility  can  be  questioned  on 
the  current  contract  with  no  award  made.  With  its  use  as  a  factor  along  with  price, 
however,  a  new  dynamic  is  present,  one  of  levels  of  risk.  By  taking  this  step  more 
subjectivity  comes  into  play.  A  determination  of  level  of  goodness  must  be  assessed, 
which  provides  more  opportunity  for  greater  disagreement.  Much  of  this  disagreement 
arises  out  of: 

•  The  measurement  parameters  used  to  assess  past  performance  data 

•  The  assessment  of  risk  with  newly  merged  companies  or  contractors  with 
no  past  performance  history 

•  Number  of  rating  levels  used 

•  Completeness  of  data  and  the  adjustments  for  governments  caused  delays 

Two  of  the  research  questions  asked:  “What  organizations  gather  and  use  past 

performance  data?”  and  “What  automated  databases  and  tools  are  used?”. 

The  PPIMS  past  performance  system  requires  that  the  administrators  of  the 
contracts  that  require  reporting  input  PAR  reports  on  contractor  performance.  Most  major 
commands  have  administrators  that  oversee  this  process.  Past  performance  data  is 
primarily  used  by  the  contracting  officers,  preaward  survey  cells,  and  source  selection 
teams  engaged  in  the  awards  of  contracts.  Reduction  in  workforce  levels  as  well  as  an 
emphasis  in  reduced  award  cycle  times  has  hindered  its  collection  and  use. 

Past  performance  data  has  many  facets  that  must  be  properly  addressed  for  its 
effective  use.  A  myriad  of  databases  can  be  used  and  the  information  must  be  understood 
as  to  how  and  what  is  measured.  PPIMS  and  the  PPIRS  database  are  designed  to  help 
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collect  data  on  higher  dollar  value  contracts,  but  these  databases  still  suffer  from 
ineomplete  data  and  ineonsisteneies  in  the  measurement  of  past  performanee  data. 
Contraetor  verification  of  this  information  must  be  made  prior  to  its  use  in  the  souree 
seleetion.  Although  efforts  have  been  made  to  automate  the  eolleetion  of  past 
performance  data,  the  process  still  is  labor  intense.  Problems  still  exists  with  negative 
data  on  eontraetors  that  is  not  always  reviewed  by  the  eontractor  prior  to  its  use.  A  more 
effective  data  eolleetion  process  needs  to  be  aehieved.  The  contraeting  personnel  and 
eontraetors  need  a  process  that  standardizes  the  method  of  measurement,  automates  its 
eolleetion,  and  allows  for  viewing  aeross  the  spectrum  of  past  performance  data  on  eaeh 
eontraetor. 

Many  of  the  difficulties  outlined  above  have  limited  the  use  of  past  performance 
to  help  ehoose  best  value  eontractor  instead  of  merely  the  lowest  priced  responsible 
eontraetor.  New  methods  of  past  performanee  use  and  data  eolleetion  can  help  achieve 
this  result.  A  new  approaeh  in  looking  into  how  and  when  it  is  used  is  instrumental  to  its 
usage  in  the  source  seleetion  proeess. 

D,  RECOMMENDATIONS 

The  final  two  researeh  questions:  “What  are  some  possible  methods  of  past 
performanee  use  in  the  future?”  and  “How  ean  the  Army  best  use  past  performance  in  the 
souree  seleetion  process?”  will  be  addressed  in  this  seetion  sinee  they  deal  with  past 
performanees  future  use  and  its  best  use  in  the  souree  selection  process. 

The  overwhelming  response  was  that  past  performance  should  only  be  considered 
as  a  source  seleetion  faetor  when  conditions  warranted  its  use.  Past  performance  should 
primarily  be  considered  when  poor  performing  eontraetors  are  anticipated  to  be  bidding, 
and  when  a  determination  nonresponsibity  cannot  be  justified  but  performanee  is  not  as 
good  as  the  eommand  would  like. 

Several  steps  must  happen  before  past  performanee  can  play  the  role  that  it  was 
envisioned  with  its  inclusion  as  a  factor  along  with  price  in  the  souree  selection  process. 
First,  the  institutional  fixes  to  gather  information  more  effectively  must  be  addressed. 
Areas  sueh  as  how  past  performanee  is  measured  and  gathered  should  be  standardized 
aeross  DoD.  Efforts  should  be  made  to  inelude  the  eontraetor  community  into  this 
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process  to  gain  both  understanding  and  buy-in  to  the  new  standardized  process. 
Automation  should  be  leveraged  in  both  the  reporting  of  performance  but  also  in  the 
scoring  of  data  and  the  review  of  performance  information  on  individual  contractors. 

Past  performance  information  should  be  considered  by  the  source  selection 
official  as  a  sub  factor  in  an  overall  capability  risk  assessment.  This  approach  speaks  to 
the  very  core  of  what  eaeh  requirer  truly  wants  to  know:  “What  is  the  risk  of  this 
company  performing  on  my  aequisition?” 

Past  performance  will  always  have  minimal,  if  any,  impact  in  the  evaluation 
process  without  a  direct  connection  to  a  contractor’s  capabilities  on  current  award.  That 
is,  it  matters  little  if  the  contraetor  has  performed  successfully  on  past  awards  if  that 
performance  is  not  direetly  related  to  the  conditions  surrounding  the  performance 
required  for  the  acquisition.  By  using  this  risk  assessment  approach  factors  such  as 
production  rates,  time  period  of  performance,  and  other  capacity  issues  can  be  assessed 
along  with  the  contractor’s  performance  history. 

E,  AREAS  OF  FURTHER  RESEARCH 

Several  areas  would  benefit  from  further  researeh  into  the  past  performance 
proeess. 

Studies  need  to  be  conducted  on: 

•  The  automation  problems  associated  with  capturing  past  performance  data 
needs  to  be  examined.  Such  research  should  focus  on  ways  to  meet  the 
true  needs  of  the  acquisition  community.  Emphasis  on  automation 
enhancements  for  the  measurement  of  data  should  be  explored. 

•  Further  research  as  to  methods  such  as  the  capability  risk  assessment 
proposed  needs  to  be  explored.  For  past  performance  data  to  become 
more  useful  in  the  future,  it  must  be  relevant  and  timely  in  the  source 
selection  process.  A  “one  size  fits  all”  approach  should  be  abandoned  to 
allow  for  a  tailoring  of  how  past  performance  data  is  used.  The  end  result 
should  be  procurement  that  provides  balance  among  the  most  capable 
contractor,  the  lowest  price,  and  lowest  performance  risk.  Consequently, 
methods  to  determine  capability  risk  assessments  should  be  explored. 

•  Further  researeh  on  methods  to  collect  and  use  past  performance  data  t 
need  to  be  conducted.  The  data  should  be  centralized  and  standardized  in 
a  true  web  based  environment  in  whieh  the  contracting  community  as  well 
as  contraetors  can  easily  assess. 
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APPENDIX  A.  FAR  CLAUSES  WITH  RESPECT  TO  PAST 

PERFORMANCE 


15.304  —  Evaluation  Factors  and  Significant  Subfactors. 

(i)  Except  as  set  forth  in  paragraph  (c)(3)(iv)  of  this  section,  past 
performance  shall  be  evaluated  in  all  source  selections  for  negotiated 
competitive  acquisitions  expected  to  exceed  $1,000,000. 

(ii)  Except  as  set  forth  in  paragraph  (c)(3)(iv)  of  this  section,  past 
performance  shall  be  evaluated  in  all  source  selections  for  negotiated 
competitive  acquisitions  issued  on  or  after  January  1,  1999,  for 
acquisitions  expected  to  exceed  $100,000.  Agencies  should  develop 
phase-in  schedules  that  meet  or  exceed  this  schedule. 

(in)  For  solicitations  involving  bundling  that  offer  a  significant 
opportunity  for  subcontracting,  the  contracting  officer  must  include  a 
factor  to  evaluate  past  performance  indicating  the  extent  to  which  the 
offeror  attained  applicable  goals  for  small  business  participation  under 
contracts  that  required  subcontracting  plans  (15  U.S.C.637(d)(4)(G)(ii)). 

(iv)  Past  performance  need  not  be  evaluated  if  the  contracting  officer 
documents  the  reason  past  performance  is  not  an  appropriate  evaluation 
factor  for  the  acquisition. 

(4)  The  extent  of  participation  of  small  disadvantaged  business  concerns 
in  performance  of  the  contract  shall  be  evaluated  in  unrestricted 
acquisitions  expected  to  exceed  $500,000  ($1,000,000  for  construction) 
subject  to  certain  limitations  (see  19.201  and  19.1202). 

(5)  For  solicitations  involving  bundling  that  offer  a  significant  opportunity 
for  subcontracting,  the  contracting  officer  must  include  proposed  small 
business  subcontracting  participation  in  the  subcontracting  plan  as  an 
evaluation  factor  (15  U.S.C.637(d)(4)(G)(i)). 

(d)  All  factors  and  significant  subfactors  that  will  affect  contract  award 
and  their  relative  importance  shall  be  stated  clearly  in  the  solicitation 

(10  U.S.C.2305(a)(2)(A)(i)  and  41  U.S.C.253a(b)(l)(A))  (see  15.204-5(c)). 
The  rating  method  need  not  be  disclosed  in  the  solicitation.  The  general 
approach  for  evaluating  past  performance  information  shall  be  described. 

(e)  The  solicitation  shall  also  state,  at  a  minimum,  whether  all 
evaluation  factors  other  than  cost  or  price,  when  combined,  are  — 
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(1)  Significantly  more  important  than  cost  or  price; 


(2)  Approximately  equal  to  cost  or  price;  or 

(3)  Significantly  less  important  than  cost  or  price  (10 
U.S.C.2305(a)(3)(A)(iii)  and  41  U.S.C.253a(c)(l)(C)). 

42.1501  —  General. 


Past  performance  information  is  relevant  information,  for  future  source 
selection  purposes,  regarding  a  contractor’s  actions  under  previously  awarded 
contracts.  It  includes,  for  example,  the  contractor’s  record  of  conforming  to 
contract  requirements  and  to  standards  of  good  workmanship;  the  contractor’s 
record  of  forecasting  and  controlling  costs;  the  contractor’s  adherence  to 
contract  schedules,  including  the  administrative  aspects  of  performance;  the 
contractor ’s  history  of  reasonable  and  cooperative  behavior  and  commitment  to 
customer  satisfaction;  and  generally,  the  contractor’s  business-like  concern  for 
the  interest  of  the  customer. 


42.1502  -  Policy. 

(a)  Except  as  provided  in  paragraph  (b)  of  this  section,  agencies  shall  prepare  an 
evaluation  of  contractor  performance  for  each  contract  in  excess  of  $1,000,000 
(regardless  of  the  date  of  contract  award)  and  for  each  contract  in  excess  of 
$100,000  beginning  not  later  than  January  1,  1998  (regardless  of  the  date  of 
contract  award),  at  the  time  the  work  under  the  contract  is  completed.  In 
addition,  interim  evaluations  should  be  prepared  as  specified  by  the  agencies  to 
provide  current  information  for  source  selection  purposes,  for  contracts  with  a 
period  of  performance,  including  options,  exceeding  one  year.  This  evaluation  is 
generally  for  the  entity,  division,  or  unit  that  performed  the  contract.  The  content 
and  format  of  performance  evaluations  shall  be  established  in  accordance  with 
agency  procedures  and  should  be  tailored  to  the  size,  content,  and  complexity  of 
the  contractual  requirements. 

(b)  Agencies  shall  not  evaluate  performance  for  contracts  awarded  under 
Subparts  8.6  and  8.7.  Agencies  shall  evaluate  construction  contractor 
performance  and  architect/engineer  contractor  performance  in  accordance  with 
36.201  and  36.604,  respectively. 

42.1503  —  Procedures. 


(a)  Agency  procedures  for  the  past  performance  evaluation  system  shall  generally 
provide  for  input  to  the  evaluations  from  the  technical  office,  contracting  office 
and,  where  appropriate,  end  users  of  the  product  or  service. 
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(b)  Agency  evaluations  of  contractor  performance  prepared  under  this  subpart 
shall  be  provided  to  the  contractor  as  soon  as  practicable  after  completion  of  the 
evaluation.  Contractors  shall  be  given  a  minimum  of  30  days  to  submit 
comments,  rebutting  statements,  or  additional  information.  Agencies  shall 
provide  for  review  at  a  level  above  the  contracting  officer  to  consider 
disagreements  between  the  parties  regarding  the  evaluation.  The  ultimate 
conclusion  on  the  performance  evaluation  is  a  decision  of  the  contracting 
agency.  Copies  of  the  evaluation,  contractor  response,  and  review  comments,  if 
any,  shall  be  retained  as  part  of  the  evaluation.  These  evaluations  may  be  used  to 
support  future  award  decisions,  and  should  therefore  be  marked  "Source 
Selection  Information  ".  The  completed  evaluation  shall  not  be  released  to  other 
than  Government  personnel  and  the  contractor  whose  performance  is  being 
evaluated  during  the  period  the  information  may  be  used  to  provide  source 
selection  information.  Disclosure  of  such  information  could  cause  harm  both  to 
the  commercial  interest  of  the  Government  and  to  the  competitive  position  of  the 
contractor  being  evaluated  as  well  as  impede  the  ejficiency  of  Government 
operations.  Evaluations  used  in  determining  award  or  incentive  fee  payments  may 
also  be  used  to  satisfy  the  requirements  of  this  subpart.  A  copy  of  the  annual  or 
final  past  performance  evaluation  shall  be  provided  to  the  contractor  as  soon  as  it 
is  finalized. 

(c)  Departments  and  agencies  shall  share  past  performance  information  with 
other  departments  and  agencies  when  requested  to  support  future  award 
decisions.  The  information  may  be  provided  through  interview  and/or  by  sending 
the  evaluation  and  comment  documents  to  the  requesting  source  selection  official. 

(d)  Any  past  performance  information  systems,  including  automated  systems,  used 
for  maintaining  contractor  performance  information  and/or  evaluations  should 
include  appropriate  management  and  technical  controls  to  ensure  that  only 
authorized  personnel  have  access  to  the  data. 

(e)  The  past  performance  information  shall  not  be  retained  to  provide  source 
selection  than  three  years  after  completion  of  contract 
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APPENDIX  B.  QUESTIONNAIRE  USED  FOR  INITIAL  SURVEY  OF 
PROSPECTIVE  INDIVIDUALS  USING  PAST  PERFORMANCE 

DATA 


My  name  is  Joseph  Tappel;  I  am  a  graduate  student  with  the  Naval  Postgraduate  Sehool 
working  on  a  master  in  eontraeting.  As  part  of  my  thesis,  I  am  eolleeting  data  on  the  use 
of  past  performanee  within  DOD  and  partieularly  within  the  Army.  In  my  fulltime  job,  I 
am  an  Industrial  Speeialist  at  AMC  Headquarters.  Your  assistanee  in  gathering  this 
information  is  greatly  appreeiated  and  eould  help  influenee  future  use  of  past 
performanee.  My  work  number  is  (703)  617-8270  of  DSN  767-8270.  My  FAX  number 
is  (703)  617-2235.  My  e-mail  is  tapp  elj  @hq  ame .  army  .mil . 

Thanks  for  your  help  gathering  this  information. 


Date 


Name 

E-mail 

Position 

Phone 

Agency 

Fax 

What  are  the  primary  ways  past  performance  is  used  in  your  organization? 

Check  all  that  apply: 

□  For  responsibility  determination 

□  As  a  stand  alone  souree  seleetion  faetor  (along  with  priee  and  other 
faetors) 

□  As  a  sub  element  in  a  souree  seleetion 

□  Not  at  all 

Who  determines  how  past  performance  will  he  used  in  your  organization? 

□  Contracting  officer 

□  Policy  guidance  within  organization 

□  Contract  specialist 

□  Team  approach 

□  Others 
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What  automated  systems  do  you  use  to  capture  past  performance  data? 

Check  all  that  apply: 

□  PPIMS 

□  CPARS 

□  DCMA  Databases 

□  CCSS  internal  data 

□  Proposal  data 

□  Others 


Of  the  past  performance  data  used  how  would  you  rate  it  effectiveness? 

Effeetiveness 


Least  Most 


□ 

PPIMS 

1 

2 

3 

4 

5 

□ 

CPARS 

1 

2 

3 

4 

5 

□ 

DCMA  Databases 

1 

2 

3 

4 

5 

□ 

CCSS  internal  data 

1 

2 

3 

4 

5 

□ 

Proposal  data 

1 

2 

3 

4 

5 

□ 

Others 

1 

2 

3 

4 

5 

What  are  some  of  the  problems  that  you  have  experienced  with  past  performance  data? 

Check  all  that  apply: 

□  Completeness  of  the  data 

□  Labor  intensive 

□  Determining  cause  for  poor  performance 

□  Validation  of  data  with  contractors 

□  Rating  performance  on  newly  formed  or  merged  companies 

□  Relative  weight  past  performance  given  in  relation  with  other  source  selection 
factors 

□  Other  problems _ 

What  is  your  role  in  past  performance? 

Check  all  that  apply: 

□  User  of  information 

□  Performance  Risk  Assessment  Group  PRAG 

□  Contracting  Officer 

□  Assessment  Group  (Preaward  Office) 

□  Gatherer/maintainer  of  information 

□  PPIMS 

□  Shipment  Data 
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□  Provide  necessary  updates  to  databases  (i.e.  shipment  and  modification 
data) 

□  Policy 

□  Automation 
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APPENDIX  C.  INTERVIEW  FORMAT  USED  FOR  ACTIVE  USERS 
OF  PAST  PERFORMANCE  DATA 


Interview  Format 
Individual  interviewed 

Position _ 

Contact  info _ 

When 


1,  What  automated  tools  are  currently  utilized? 

2,  How  often  is  past  performance  used  as  a  source  selection  factor? 

3,  How  are  you  implementing  the  current  Federal  Procurement  Policy’s 
guidance  on  the  use  of  past  performance? 

4,  What  changes  are  necessary  to  fully  comply? 

5,  What  do  you  think  the  role  of  past  performance  should  he? 
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